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TEST WISENESS ABILITIES ON PERSONALITY SCALES WERE 
MEASURED BY SPECIALLY DEVELOPED INSTRUMENTS. THE FOUR 
TEST-WISENESS VARIABLES WERE— (1) ESTIMATING DESIRABILITY 
(ACCURACY IN ESTIMATING THE DESIRABILITY OF PERSONALITY 
ITEMS), (2) ESTIMATING COMMUNALITY (ACCURACY IN ESTIMATING 
THE COMMUNALITY OF ITEMS), AND (3) ABILITY TO IDENTIFY ITEMS 
(CORRECTNESS IN IDENTIFYING AND "KEYING" ITEMS). SEVERAL ROLE 
PLAYING MEASURES BASED ON THE DIFFERENCE BETWEEN SCORES 
OBTAINED WITH FAKE-GOOD AND FAKE-BAD INSTRUCTIONS ON EACH OF 
FOUR SCALES FROM THE GUILFORD-ZIMMERNAN TEMPERAMENT SURVEY 
WERE ALSO ADMINISTERED. TESTS OF VERBAL COMPREHENSION, 

GENERAL REASONING, AND INDUCTION, MEASURES OF CATEGORY WIDTH 
AND EQUIVALENCE RANGE ON AN INVENTORY OF SOCIAL DESIRABILITY 
REPONSE STYLE, AND DEFENSIVENESS MEASURES WERE USED IN THIS 
STUDY. THE SUBJECTS WERE NINETY-ONE UNDERGRADUATE WOMEN. THE 
RESULTS INDICATED A HIGH LEVEL OF TEST-WISENESS. HOWEVER, 
TEST-WISENESS WAS NOT A BROAD GENERAL ABILITY BUT CONSISTS OF 
A SET OF DISTINCT AND UNRELATED ABILITIES. TEST-WISENESS WAS 
NOT FOUND TO BE THE MAJOR CAUSE OF DISSEMBLING. THIS PAPER 
WAS PRESENTED AT THE AMERICAN PSYCHOLOGICAL ASSOCIATION 
MEETING, (WASHINGTON, D.C., SEPTEMBER 1967). (CG) 
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"Test-Wiseness" on Personality Scales^ 

According to test lore, people vary in their knowledge about tests, and 
this "test-wiseness" affects their performance on these devices* It is be- 
lieved that test-wise individuals obtain higher scores on ability and aptitude 
tests, and distort their scores on personality inventories (Anastasi, I 96 I; 
Cronbach, I96O; Ebel & Damrin, I96O; Fishman, Deutsch, Kogan, North, & Whiteman, 
1964; Goslin, 1963; Guilford, 1959; Pauck, 1950; Thorndike, 19^9; Vernon, 1958, 
1962), Despite the prevalence of these notions, the relevant data are sparse 
(Millman, Bishop, & Ebel, 1965)^ 

Several abilities may be involved in test-wiseness on personality inven- 
tories. One is the ability to respond to an inventory in a way that is con- 
sistent with a prescribed role* This particular kind of "impression management" 
(Goffman, 1959) is displayed in role-playing studies (Ellis, 1953; Waters, I 965 ), 
which administer personality scales with instructions to "fake" "good" or "bad" 
roles . 

Another potentially relevant ability is accuracy in estimating the desir- 
ability of personality items,, Edwards' (1957) social desirability paradigm 
suggests that accurate knowledge of desirability is needed for socially desirable 
responding and, more generally, for dissembling on personality inventories# 
Partially supporting such a conception, Wiggins (1966) found that subjects* ac- 
curacy in estimating the average desirability ratings made by others did cor- 
relate with skill in faking-good on MMPI (Hathaway & McKinley, 1951) clinical 

scales# However, this accuracy did not correlate with success in faking-good 

2 

on MMPI scales measuring test-taking attitudes* 
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A similar ability is accuracy in estimating the "comiixunality” (Wiggins, 
1962) or frequency of endorsement of personality items. In Wiggins' (1966) 
study, this accuracy did not correlate with ability to fake-good on any of 
the MMPI scales. 

Accuracy in analyzing a personality scale and determining the na.-:ure of 
the traits it is intended to measure may also be a pertinent ability. This 
ability has not been investigated in the context of test-wiseness. 

The present study was designed to explore the role of these test- 
wiseness abilities on personality scales, using specially developed measures 
of these skills. 



Method 

3 

Four kinds of test-wiseness variables were employed. One, called 
.Estimating Desirability, assessed accuracy in estimating the desirability 
of personality items. Estimating Communality, which was a similar instru- 
ment, gauged accuracy in estimating items' communality. Ability to Identify 
Items was a measure of correctness in identifying and "keying" items loading 
the same factor. There were several role-playing measures. They were based 
on the difference between scores obtained with fake-good and fake-bad 
instructions on each of four scales from the Guilford-Zimmerman Tempera- 
ment Survey (GZTS, Guilford & Zimmerman, 19 ^ 9 )* The scales were General 
Activity, Sociability, Emotional Stability, and Personal Relations . They were 
given with standard instructions, as well as with fake-good and fake-bad 
instructions. Several other instruments were administered. These were tests 
of verbal comprehension, general reasoning, and induction; measures of cate- 
gory width and equivalence range; and an inventory containing social 
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desirability (SD) response style and defensiveness measures. The subjects were 
91 undergraduate women. 



Results 



In Thble 1, comparisons of the mean scores on the test-wiseness measures 
with their theoretical limits indicate that the subjects were highly test- 
wise in absolute terms. The means for Ability to Identify Items and for the 
role-playing measures were particularly close to their theoretical limits. 

Insert Table 1 about here 

In the analyses that follow, the correlations for the role-playing mea- 
sures have been reflected in sign so that, in effect, high scores on all test- 
wiseness measures represent high ability. In Table 2, none of the correlations 
between the four kinds of test-wiseness measures was significant (p > .. 05 ), 
except for the ,*-3 correlation between Estimating Desirability and Estimating 
Communality, The role-playing measures generally correlated significantly 
with each other, but the correlations were moderate. 

The test-wiseness measures had some significant but moderate correlations 
with the standard GZTS scales in this table. All but one of the correlations 
of Estimating Desirability, Estimating Communality, and Ability to Identify 
Items were with the Emotional Stability scale. In general, the role-playing 
measures correlated with the standard version of their own scale, but they did 
not correlate with the other GZTS scales. 

Insert Table 2 about here 
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In TeXile 3, Estimating Desirability, Estimating Communality, and Ability 
to Identify Items consistently had significant correlations with the SD mea- 
sures. The two accuracy instruments were positively related to socially 
desirable responding, but Ability to Identify Items was negatively related 
to this response style. In contrast to these extensive correlations, only 
one correlation between these test-wiseness measures and the defensiveness 
scales was significant. 

Virtually all the significant correlations for the role-playing mea- 
sures in this table involved the Bnotional Stability and Personal Relations 
scales, on the one hand, and the SD scales, on the other. Role-playing suc- 
cess on these GETS scales was positively related to socially desirable re- 
sponding. Like the other test-wiseness instruments, the role-playing mea- 
sures had few correlations with the defensiveness scales. 

Incidentally, none of the test-wiseness measures correlated significantly 
with any of the ability tests, and they had only scattered correlations with 
the cognitive style measures. 

Among the standard GZTS scales. Sociability, Qnotional Stability, and 
Personal Relations correlated significantly with all the SD measures in this 
same table. Hiese GZTS scales were positively related to socially desirable 
responding. They also had relatively low and less consistent correlations 
with the defensiveness scales, the GZTS scales being positively related to 
defensiveness . 

Insert Table 3 about here 



o 
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Discussion 

The high level of test-wiseness was striking. Equally important was 
the specificity of the test-wiseness measures. The different kinds of 
instruments were unrelated, with one minor exception, though there was some 
generality among the role-playing variables • This finding implies that test- 
wiseness is not a broad, general ability, but consists of a set of distinct 
and largely unrelated skills. 

The subjects had the ability to distort their scores on personality 
scales, as gauged from the extent of test-wiseness that was observed. And 
considerable dissembling apparently did occur, as suggested by the correla- 
tions of the response style measures with the GZTS personality scales. But 
the limited relationships between the test-wiseness measures and the person- 
ality scales imply that test-wiseness was not a major cause of this dissembling. 
Why wasn’t test-wiseness closely linked with the distortion on these scales? 

If test-wiseness didn’t produce the dissembling, what did? One plausible 
answer is that test-wiseness did, indeed, produce the distortion, but this 
relationship was obscured by the heterogeneity of the subjects. They may 
liave consisted of those unmotivated to distort, those motivated to distort 
in a favorable direction, and those motivated to distort in an unfavorrible 
direction. The test-wiseness measures would be expected to correlate with 
the personality scales in the two motivated groups, but in opposite direc- 
tions, and no correlation would be anticipated in the unmotivated group. 

Hence, the overall correlations for the total group would be relatively small. 
Research on this issue is under way. If, in fact, test-wiseness was not the 
major source of the distortion, no compelling explanation can be offered for 
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vhy this should be so or for the real source of the distortion. One specu- 
lation is that dissembling is largely unconscious, and, thus, not dependent 
on the skills involved in the test-viseness measures. 

The consistent correlations of the test-viseness measures with the SD 
scales contrast sharply with the scanty correlations of the test-wiseness 
measures with the defensiveness scales. The explanation may lie in differ- 
ences in the item composition of the two kinds of scales. Unlike most per- 
sonality items, including those on the SD scales, the items on the defensive- 
ness scales typically have discrepant decirability and communality (Jackson 
& Messick, 1962). The desirable items are seldom endorsed (e.g., "I read 
in the Bible several times a week") and the undesirable ones are often 
endorsed (e.g., "At times I feel like swearing"). The complex and unusual 
interaction of desirability and communality on the defensiveness items, their 
rareness, and their underrepresentation on the test-wiseness measures should 
attenuate the correlations of the defensiveness scales with the test-wiseness 
variables. A different explanation of these findings is that the defensive- 
ness scales may .tap a characteristic that is under less conscious control 
than the trait measured by the SD scales. Socially desirable responding may 
entail a relatively systematic consideration of the items' desirability and 
communality, accounting for its link with the accuracy measures . Defensive- 
ness, because it may not involve such considerations, is unrelated to 
knowledge of these item characteristics. 
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Footnotes 

^T]his study was supported ty the National Institute of Child Health and 
Human Development, under Research Grant 1 POl HD 01762 — 01 * Thanks are due 
to Henrietta Galla^er for supervising the statistical analyses. 

^ese results were obtained with an "absolute accuracy" score, which 
is rou^y analogous to the scores used in the present study. 

^Estimating Desirability employed 95 heterogeneous MMPI items. The 
subject judged the social desirability of each on a nine-point scale. The 
score was the product-moment correlation, transformed to Fisher's z, between 
his judgments for the items and the items' social desirability scaj.e values 
(Messick & Jackson, I 961 ) . 

Estimating Communality employed 95 other MMK items. Using a nine- 
point scale, the subject judged how frequently college students responded 
"true" to each item. The score was the transformed correlation between his 
judgments and the items' actual endorsement frequencies for I 90 Stanford 
undergraduates (Wiggins, 1959)* 

Ability to Identify items consisted of three similarly constructed sub- 
tests, each based on a different published factor analysis of personality 
items (Comrey & Souf i, I960, I 96 I; Layman, 19^0) . A subtest consisted 
of 15 items; the ei^t with the hipest loading on the same factor, and 
seven others— each with the hipest loading on one of seven other factors. 
The subject was instructed to identify the items that refer to the same 
personality trait and to "key" them ("true" or "false"). The score was 
the number of items from the main factor that were identified as in- 
volving the same trait and '•.orrectly keyed, plus the number of items not 




from that factor which the subject indicated did not refer to the same trait. 
A total score for the instrument was obtained^ weighting tiie subtest scores 
for maximal reliability (Green, 1950). 

The role-playing measures en^loyed a difference score (McNemar, 1953, 
p. 48), based on the fake-bad score minus the fake-good score. The role- 
playing instructions were adapted from those for the college admission situa- 
tion used by Yonge and Heist ( 1965 ) • 
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Table 1 

Means, Standard Deviations, and Theoretical Score 
Limits of Test-Wiseness Measures 









Theoretical Score Limits 


Measure 


Mean 


S.D. 


Minimum 

Test-Wiseness 


Maximum 

Test-Wiseness 


Estimating Desirability 


1.08 


•16 


-3.00 


3.00 


Estimating Communality 


.68 


.27 


-3.00 


3.00 


Ability to Identify Items 


15.86 


1.20 


,00 


19.39 


Role-Playing — General Activity 


- 10,55 


5.72 


28.48 


-21.93 


Role-Playing — Sociability 


- 13 .35 


5.71 


30.18 


- 18 . 7 ** 


Role-Playing— Emotional Stability 


-18.56 


4.31 


25.77 


- 2 if .37 


Role-Playing — Personal Relations 


-17.33 


4,13 


21.47 


- 2 l ^.50 
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